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The Center 



The mission of the Center for Research on Effective Schooling for Disadvantaged Students 
(CDS) is to significantly improve the education of disadvantaged students at each level of 
schooling through new knowledge and practices produced by thorough scientific study and 
evaluation. The Center conducts its research in four program areas: The Early and Elementary 
Education Program, The Middle Grades and High Schools Program, the Language Minority 
Program, and Sie School, Family, and Community Connections Program. 

The Early and Elementary Education Program 

This program is working to develop, evaluate, and disseminate instructional programs 
capable of bringing disadvantaged students to high levels of achievement, particularly in the 
fundamental areas of reading, writing, and mathematics. The goal is to expand the range of 
effective alternatives which schools may use under Chapter 1 and other compensatory education 
funding and to study issues of direct relevance to federal, state, and local policy on education of 
disadvantaged students. 

The Middle Grades and High Schools Program 

This program is conducting research syntheses, survey analyses, and field studies in middle 
and high schools. The three types of projects move from basic research to useful practice. 
Syntheses compile and analyze existing knowledge about effective educaiion of disadvantaged 
students. Survey analyses identify and describe current programs, practices, and trends in middle 
and high schools, and allow studies of their effects. Field studies are conducted in collaboration 
with school staffs to develop and evaluate effective programs and practices. 

The Language Minority Program 

This program lepresents a collaborative effort. The University of Califomia at Santa 
Barbara is focusing on the education of Mexican- American students in Califomia and Texas; 
studies of dropout among children of recent immigrants are being conducted in San Diego and 
Miami by Johns Hopkins, and evaluations of learning strategies in schools serving Navajo 
Indians are being conducted by the University of Northern Arizona. The goal of the program is 
to identify, develop, and evaluate effective programs for disadvantaged Hispanic, American 
Indian, Southeast Asian, and other language minority children. 

The School, Family, and Community Connections Program 

This program is focusing on the key connections between schools and families and between 
schools and communities to build better educational programs for disadvantaged children an& 
youth. Initial work is seeking to provide a research base concerning the most effective ways for 
scnools to interact with and assist parents of disadvantaged students and interact with the 
community to produce effective community involvement. 
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Abstract 



This report summarizes an evaluation of a staff development program to reduce 
disparity in educational achievement across race and sex. Teacher Expectations and 
Student Achievement (TESA; Kerman, Kimball, & Martin, 1980) was implemented in 
an elementary school. Achievement test scores, attitudes towards school and self, 
perceptions of teacher practices, and grade retentions were compared for the 306 
students of teachers who volunteered to participate in the experimental program and 
students in the same school (n=329) and in a different school (n=ii50) whose teachers 
did not participate. The results differ depending upon which comparison group is 
used. A small positive effect is implied by the within-school comparison, and a 
negative effect is implied by the between-school comparison. 



Increasing Teacher Expectations for Student Achievement: 

An Evaluation 



Many urban school districts face high 
rates of student failure and drop-out. This 
report summarizes the results of a study of 
one component of a comprehensive program 
developed by educators and researchers 
working together to reduce drop-out in the 
Charleston County School District (CCSD) 
public schools (Gottfredson & Gottfredson, 
1990). It illustrates some early risk factors 
for drop-out as experienced in this school 
system, describes a staff development pro- 
gram aimed at ameliorating early risk factors 
for drop-out, and presents the results of an 
evaluation of the program. 

Being held back in a grade strongly 
predicts school drop-out (Bachman, Green, 
& Wirtanen, 1971). Figure 1 shows the per- 
centage of first grade students who were 
enrolled in CCSD in 1982 who progressed 
normally and so were in the grade expected 
six years later ("on-grade"), were behind at 
least one grade, or were no longer in the 
school system. Information is shown for all 
CCSD elementary schools and separately for 
the two elementary schools included in our 
study. 

The figure shows that a minimum of 
29% of the elementary school children in 
CCSD accumulate the biggest risk factor for 
dropout by the time they are at the age 
appropriate for 7th grade. It is not possible 
to determine what percentage of those who 
have left the system have been retained in 
grade, but if we assume that the distribution 
of grade retention is the same for students 
who transfer out as it is for those who stay, 
the percentage behind grade is almost 50%. 



Table 1 shows that the incidence of 
grad( retention is unequally distributed 
across race and sex groups. For all CCSD 
elementary and middle schools, African- 
American students were retained in grade at 
the end of the 1988-89 school year at ap- 
proximately twice the rate of white students, 
and African- American males were particular- 
ly adversely affected. The two elementary 
schools included in our study had higher 
than average ratios of African-American to 
white grade retentions. For males, the ratios 
were 4.2 and 2.8 African-American student 
retentions to one white student retention. 

Table 2 shows estimates of the preva- 
lence of retention - the effect of accumulat- 
ed grade retentions over the elementary 
school years. The table shows the percent- 
age of each race and sex group who were 
"overage" in Spring, 1989. In this table, 
students are coded as overage if their age 
was greater than the highest age possible for 
a student who entered tlx CCSD system and 
progressed normally througii the grades. In 
the schools participating in our study, 47% 
and 62% of the African-American males 
were overage. The percentage of African- 
American males overage was 2.5 to 3 times 
greater than the comparable percentage for 
white males. The African-American/white 
discrepancy was even greater for females. 
Educational success is unequally distributed 
across race and sex groups in the CCSD 
schools. 

Why is educational success so unevenly 
distributed? Among the possible reasons are 
differences in the early childhood experienc- 
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es that prepare children for school, in char- 
acteristics and abilities that are rewarded and 
punished in school (e.g., aggressive behavior, 
intellectual capability), and in teachers' 
expectations for student achievement that are 
translated into specific teacher practices and 
that affect student outcomes. This study 
explores the latter possibility by examining 
the effects of a staff development program 
aimed at making teachers aware of their own 
inequitable treatment of low achievers and 
providing them with new techniques for re- 
sponding to students. 



Research on Teacher Expectations 

Our society's expectations for education 
have changed dramatically over the years. 
Carter and Klotz (1990) identified three 
phases of societal expectancies (p. 39-40): 

1. 1837-1906 - Schools were expected 
to teach the 3 R's and self-discipline and to 
produce moral citizens. 

2. 1910-1975 - The nonnal cur/e was 
an expected product of instruction; teachers 
were accountable to provide an opportunity 
to learn. 

3. 1976-present - Beginning with Ben- 
jamin Bloom's mastery learning, schools 
have been expected to teach all students the 
basics and teachers are more accountable for 
learning. 

As the expectation that all children 
should leam has increased, and as large 
group differences in school success rates 
have become more apparent, attention has 
turned to teachers' potential to influence stu- 
dent learning through their expectations and 
behaviors. 



In Pygmalion in the Classroom, Rosen- 
thal and Jacobson (1968) hypothesised that 
teachers form expectations for student per- 
formance, tiiat students respond to the be- 
havioral cues of their teachers, and that 
student performance is shaped by these 
expectations. This hypothesized expectancy 
effect became a central topic of process- 
product research as investigators attempted 
to test the Pygmalion hypothesis by observ- 
ing behavior in naturally occurring class- 
rooms (Wittrock, 1986). 

Brophy and Good (1970) suggested that 
teachers may differentiate their behavior 
toward students based on their expectations, 
and that students may respond to teachers' 
behavioral cues and alter their self-concept 
and achievement motivation to conform to 
teachers' expectations. 

Researchers have explored each compo- 
nent of the expectancy model: the accuracy 
of teacher expectations, the variation in 
teacher behaviors based on their expecta- 
tions, and student interpretations and inter- 
nalization of teacher expectations and behav- 
iors. A summary of research related to each 
component follows. 

Teacher expectations. Teachers have 
been found to overestimate the achievement 
of high achievers, underestimate that of low 
achievers, and to be least accurate in predict- 
ing the responses of low achievers (Cola- 
darci, 1986; Hoge & Butcher, 1984; Patri- 
arca & Kragt, 1986). Babad (1985) reported 
that less experienced teachers, teachers who 
believe either that integration will result in 
great improvement or no improvement, and 
teachers who prefer the lecture method more 
often had biased expectations. 



Raudenbush (1984) examined 18 studies 
to determine if teacher expectations influ- 
enced IQ test scores. He found that the 
better teachers knew the students, the less 
potent were the effects. Also, the effects on 
test scores were larger for students in the 
early elementary grades (grades 1 and 2) and 
in grade 7 than they were for students in the 
upper elementary grades. 

Student characteristics such as physical 
attractiveness, socioeconomic status, race, 
use of standard English, and retention status 
are related to the degree of discrepancy 
between teacher expectations for academic 
success and actual achievemfint (Cecil, 1988; 
Dusek & Joseph, 1983; Kenealy, Neil, & 
Shaw, 1988; Williams & Muehl, 1978; 
Crowl, 1971; Gaines & Davis, 1990). In a 
study of teachers' attributions for student 
performance, Peterson and Barger (1984) 
found that teachers attributed the success of 
perceived high achievers to ability and that 
of perceived low achievers to luck, making 
it difficult for perceived low achievers to 
change their teachers' expectations through 
their own efforts. 

Teacher behaviors. Much research has 
attempted to identify teacher behaviors that 
are related to teacher expectations for stu- 
dent success. Good (1987) found that the 
following behaviors are used more often 
with perceived low achievers: giving gener- 
al, insincere praise; providing less frequent 
and less informative feedback; requiring less 
effort; more frequently interrupting student 
speech; paying less attention to the student; 
offering fewer opportunities to respond in 
class; reducing wait time; giving more criti- 
cism; making less eye contact; giving fewer 
smiles; having fewer public and more private 
interactions; monitoring and structuring 
activities more closely; making less use of 
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student ideas; providing fewer cues to im- 
prove responses; rewarding more incorrect 
answers or inappropriate behavior; and 
assigning seats further from the teacher. 
Rist (1972) found that teachers spent less 
time in close proximity to perceived low 
achievers. 

Student response to teacher expectations. 
From their first years in school, students are 
able . J perceive differences in teacher expec- 
tations for their own pcriormance and that of 
their peers. Wittrock (1986) summarized the 
research in this area: "The findings suggest 
the early and definite effects teachers can 
have upon students' expectations and self- 
concepts of school ability" (p. 298). Young 
students perceive that low achievers receive 
more directions, rules, work and negative 
feedback and that high achievers enjoy 
higher teacher expectations for their perfor- 
mance and more freedom of choice (Wein- 
stein, Marshall, Brattesani. & Middlestadt, 
1982). Cooper (1983) found that low-expec- 
tation students receive more non-effort- 
contingent feedback designed to control their 
behavior, consequently these students are 
less likely to develop beliefs in the value of 
effort, are less persistent, and less successful. 



Program Description 

Teacher Expectations and Student 
Achievement (TESA; Kerman, Kimball, & 
Martin, 1980) is a teacher awareness training 
program designed to reduce the negative 
effects of low teacher expectations. The 
program was eariier titled "Equal Opportu- 
nity in the Classroom." and development was 
funded by an Elementary-Secondary Educa- 
tion Act (ESEA) Title III grant. 



TESA focuses on 15 classroom behaviors 
that research implies are effective teaching 
practices and that are used more often with 
perceived high achievers than with perceived 
low achievers. The behaviors arc organized 
into three "strands" ~ response opportunities, 
feedback, and personal regard. Figure 2 
summarizes the three strands and 15 behav- 
iors targeted by TESA training. 

Teacher training is provided in five 
three-hour sessions one month apart. At 
each session, a unit focusing on one behav- 
ior from each strand is presented. Research 
on the behaviors is summarized and partici- 
pants discuss the effectiveness of the behav- 
iors and reasons why they may be more 
often used with perceived high achievers. In 
preparation for peer observation, participants 
practice observing teaching episodes and 
tallying positive and negative interactions for 
each teacher behavior. During the weeks 
between TESA lessons, the participants 
observe at least three classrooms of other 
teachers, and each is observed teaching and 
given feedback at least three times. 

During the 1973-74 school year, a study 
was conducted as part of the ESEA Title III 
evaluation. A total of 2,968 students in 
grades K - 12 and 258 teachers p irticipated 
in the study. Participating districts were 
asked to identify a comparison group total- 
ing 40 classes. Evaluators examined pre- 
and post-training teacher use of the TESA 
behaviors with identified low achievers and 
pre- and post-training California Test of 
Basic Skills (CTBS) scores for students of 
the TESA and control teachers. 

The results indicated that teachers in- 
creased their use of the 15 behaviors with 
low achievers whom they targeted. Positive 
interactions increased with these low achiev- 



ers and teachers used the new behaviors as 
frequently with low achievers as with high 
achievers. Furthermore, post-TESA CTBS 
Language Arts scores for the experimental 
group were higher than those of the control 
group (p < .05), controlling for pre-TESA 
scores. 

The validation study team reported that 
"it is clear that the educational gains in 
language arts made by the students perceive(^ 
to be low performers were statistically sig- 
nificant, educationally important, readily de- 
monstrable, and attributable to the inservice 
training provided by the project" (Kerman, 
Kimball, & Martin, 1980, p. G-8). 

Since the mid-seventies, TESA training 
has been offered in school districts across 
the nation. TESA training materials are now 
published and distributed by Phi Delta Kap- 
pa. Kerman and Associates provide trainer 
training on a regular schedule in three train- 
ing sites and serve as training consultants to 
school districts. 

According to Joyce and Showers (1988), 
TESA is "one of the most-used offerings in 
staff development programs across the coun- 
try" (p. 42). They found TESA to be mod- 
estly effective (i.e., producing experim«ntal- 
control group differences in learning ot up to 
one-half standard deviation) and concluded 
that the program is valuable since the devel- 
opers "provided a very direct avenue to 
helping teachers produce what most observ- 
ers would agree is a more desirable class- 
room climate, an avenue that can be fol- 
lowed with modest amounts of training and 
that promises general educational benefits in 
the peisonal, social, and academic domains" 
(p. 44). 



i U 



The Present Study 

Five elementary schools volunteered to 
participate in a southeastern county's TESA 
program during the 1988-89 school year. 
Two of the five were selected to participate 
in the evaluation. They were selected be- 
cause their populations were diverse and 
because their principals were agreeable to 
the evaluation. One of these two schools 
was eventually dropped from the evaluation 
because the program was not implemented 
according to the program standards and 
some of the records necessary for describing 
the level of implementation were lost during 
Hurricane Hugo. 

Two experienced teachers at the remain- 
ing TESA school were trained as TESA 
trainers. These teachers then presented the 
program to their colleagues. TESA had been 
selected as an appropriate staff development 
program by the School Based Management 
Team of the school and the school had been 
awarded a $5,000 grant for the training. The 
principal or assistant principal attended all 
training sessions as participants and provided 
consistent and enthusiastic support for imple 
mentation. The trainers were rigorous in 
their use of the training model as specified 
in the TESA trainer manual. 



Method 

Sample 

Students in the TESA school (School A) 
and a comparison school (School B) com- 
pose the study sample. The two schools are 
compared in Tables 3 and 4 using data from 
the Spring of 1988 - prior to the year dur- 
ing which TESA was implemented at School 
A. Table 3 shows that the schools were 



similar in terms of their total enrollments, 
percentages African- American and overaged 
students, percentage of students receiving 
free or reduced lunch, and achievement test 
scores. 

Table 4 shows the results of a school 
climate battery (Effective School Battery; 
ESB; Gottfrcdson, 1984) administered to 
teachers in the two schools in the Spring of 
1988. The percentile scores suggest that the 
schools were similar in terms of the percep- 
tions and characteristics of their teachers. 
Teachers in School A reported that they had 
more resources for instruction and had un- 
dergone more professional development ac- 
tivities than teachers in School B, and they 
reported feeling somewhat more secure in 
their school (i.e., they reported fewer experi- 
ences of victimization). On the other hand, 
teachers in School B reported slightly higher 
levels of job satisfaction. 

In summary, the TESA and comparison 
schools were similar in student composition, 
most teacher attitudes, and perceptions of the 
school environment. Teachers at School A 
may have been better prepared in terms of 
the resources available to them and the 
professi:^)nal devclopnient offered to them 
prior to the year during which TESA was 
implemented. 

Twenty teachers volunteered to partici- 
pate during the first year of the program. 
The staff of the school understood, however, 
that over a two- to three-year period all 
teachers would be trained in TESA. The 
trainers and school administrators indepen- 
dently rank-ordered all teachers in the school 
based on their perceptions of their effective- 
ness in producing student learning. TESA 
teachers were not significantly different on 



5 



ERIC 



a 



these rankings than those teachers who chose 
to delay participation. 

Table 5 shows the number and percent- 
age of participating students and teachers. 
Overall, approximately half of the teachers 
volunteered, and 58% of the student-; in the 
school were in these teachers' classrooms. 
Participation was not uniformly distributed 
across the grade levels. 



Measures 

Characteristics of the students prior to 
involvement in TESA were obtained from 
school records and a short survey. School 
records provided information on demograph- 
ics, special education services provided, 
grade retentions and achievement test scores 
from the 1987-88 school year.' An attitude 
survey was administered in the Fall of 1988 
to students in grades four and five only. 
Outcome measures (promotions, achievement 
test scores, and the attitude surveys) were 
collected at the end of the 1988-89 school 
year. 



The pre-treatment achievement lest for first 
graders was administered in the Fall of 1988. 

''students in grade four were excluded from the 
e dluation because (a) 96% of the fourth grade 
students in School A participated in TESA, leaving 
few fourth grade students for comparison within 
School A (sec Analysis section) and (b) a large per- 
centage of fourth graders had been held back the 
previous year, and the achievement test available from 
the previous year was different for those that had and 
had not been held back. Analyses of survey data 
were done with and without fourth graders. The 
results did not differ in the two analyses. For sim- 
plicity, the analysis including only fifth graders is 
reported. 



The analyses were performed separately 
by grade because different combinations of 
pre- and post test achievement scores were 
available for different grade levels. The fol- 
lowing 1988-89 grade groupings were used: 
grade 1, grades 2 and 3, and grade 5/ All 
incoming first graders took the Cognitive 
Skills Assessment Battery (CSAB) as a 
pretreatment achievement indicator in the 
Fall of 1988. For grades 2 and 3, the pre- 
treatment achievement indicator was the 
Basic Skills Assessment Program (BSAP) 
score from the previous Spring. The post- 
treatment achievement test administered to 
first through third griulers in the Spring of 
1989 was the BSAP. For sfudents in grade 
5 in 1988-89, the California Test of Basic 
Skills (CTBS) was administered both as a 
pre-treatment (Spring, 1988) and post-treat- 
ment achievement measure. BSAP scale 
scores for Math and Reading, CTBS scale 
scores for Math and Reading, and the total 
score from the CSAB were used. 

In Fall 1988 and Spring 1989, fifth grade 
students completed a survey designed to 
measure attachment to school, academic self- 
concept, and perceptions of teacher practices 
related to the TESA intervention. A factor 
analysis confirmed the expected three-factor 
structure. We averaged z-transformed 
items^ to form three scales, called Aca- 
demic Self Concept, Attachment to School, 
and Teacher Practices. Table A-1 lists the 
items in each factor and explains how each 
was coded. The alpha reliabilities for the 
pretest survey were .75 for the Academic 
Self-Concept scale, .78 for the Attachment to 
School scale, and .74 for the Teacher Prac- 



Items were standardized before creating the 
composite because the response formats were differ- 
ent for different scales. 
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tices scale. For the post-test survey the 
reliabilities were .79, .81, and .78, respec- 
tively. Note that higher scores on these 
scales indicate lower academic self-concept, 
lower attachment to school, and more nega- 
tive judgments about teacher practices. 

Analysis 

Students participating in TESA were 
compared with two different groups; Stu- 
dents within School A who were in classes 
of the teachers who did not participate in the 
TESA program, and students in the compar- 
ison school. School B. For the latter com- 
parison, we selected a stratified random 
sample (stratified by grade level) of students 
from School B so that ihe average grade 
level of the TESA and comparison students 
would be the same. Of the 712 students at 
School B, 249 were selected to serve as a 
comparison group for the 305 School A 
students who participated in TESA. 

Analysis of covariance (ANCOVA) was 
used to assess pui>t-treatment differences 
between the TESA group and each of the 
comparison groups. For each grade group- 
ing, if significant pre-treatment differences in 
either comparison (School A/School B or 
TESA/non-TESA within School A) were 
found, the pre-treatment variable was used ?s 
a covariate. 

Pre-treatment variables examined for 
possible group differences were: proportion 
male, proportion African American, mean 
age, proportion of students receiving free or 
reduced lunch, proportion of students in self- 
contained classes, proportion of students in 
resource classes, proportion with a handicap 
(speech, hearing, visual, orthopedic, and/or 
emotional), proportion retained in the last 



school year, and pre-treatment achievement 
test means. For fifth graders, the Fall, 1988 
scores on the three attitudes s'-^Ues were also 
examined. 

Finally, we examined interaction effects 
of initial achievement level of students with 
TESA participation to determine whether the 
effects of TESA differed for students of 
different achievement levels. TESA's em- 
phasis on equitable treatment for low achiev- 
ers should produce larger effects for low 
achievers. Each student was ranked as a 
low, medium, or high achiever, on the basis 
of scores on the pre-treatment achievement 
test. The student was coded as a high 
achiever if he or she was in ihe top third ot 
the test-score distribution for the school, 
medium if he or she scored in the middle 
third of the distribution, or low if he or she 
scored in the bottom third of the distribution 



Results 

Pre-treatment differences across the three 
groups are shown in Table 6. For all grade 
groupings, fewer TESA participants were 
receiving special education services than 
non-participants in School A, and the pro- 
portion of TESA participants retained the 
previous year was higher than for non-partic- 
ipants in School A. In grade 1, pre-TESA 
cognitive skills, as measured by the CSAB, 
were higher for TESA participants than for 
non-participants, and in grade 5, TESA 
participants were younger than non-partici- 
pants. 

With respect to the School A - School B 
contrast, a higher proportion of grade 1 
through 3 School B students and a lower 
proportion of grade 5 School B students 
were receiving special education servi<.es. In 
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addition, for grades 2, 3, and 5, a higher 
proportion of TESA students in School A 
had been held back the previous year, anc 
TESA Students had higher reading test 
scores. 

In summary, TESA participants started 
the 1988-89 school year with the disadvan- 
tage of having been retained at a higher rate 
in the previous year, but with the advantage 
of having higher reading test scores. Special 
education students also seem to have been 
under-represented in the TESA classrooms. 
These differences are statistically controlled 
in the analyses of post-test differences. 

Table 7 shows the unadjusted means for 
each outcome measure for the three groups 
of students. In this table, asterisks indicate 
significant uncontrolled mean differences, 
and superscripted letters indicate significant 
mean differences when the pre-treatment 
differences are statistically controlled. 

Table 8 shows the adjusted means for 
outcome variables for which significant 
treatment effects were found after controlling 
for pre-treatment differences. These num- 
bers give a more accurate representation of 
the true magnitude and direction of effects, 
once pre-treatment variables are controlled. 

Grade 1. Table 7 shows that TESA 
participants were promoted at a higher rate 
and performed better on the achievement 
tests than did non-participants in School A, 
but the differences were not significant. The 
post-treatment BSAP reading scale score 
advantage for TESA participants was almost 
significant (p < .10) when pre-treatment 
differences were controlled. However, a 
higher proportion of School B comparison 
students were promoted. School B students 
also had higher post-test BSAP reading and 



math scores. These differences remained 
sig- '.ficant after controls for pre-treatment 
differences were applied. Table 8 shows 
adjusted scores on these outcome variables. 

Grades 2 and 3. The results for grades 
2 and 3 are similar to those for grade 1. 
Within School A, unadjusted means show 
that TESA participants have higher promo- 
tion rates than non-participants. Both grade 
2 and grade 3 differences disappear, howev- 
er, when pre-treatment variables are con- 
trolled. The School B/School A comparison 
shows that School B students had signifi- 
cantly higher post-BSAP reading scale scores 
(p < .01), even after controlling for pre-treat- 
ment differences. Table 8 shows the mag- 
nitude of the adjusted mean difference. 

Grade 5. No significant differences were 
found for the TESA and non-TESA students 
within School A. School B students scored 
lower than TESA participants on the reading 
portion of the post-test CTBS and the differ- 
ence remained when pre-treatment variables 
were controlled (p < .01). 

Grade 5 survey. TESA participants from 
School A liked school less than their com- 
parison students at School B, and this differ- 
ence persisted after controlling for differ- 
ences in pre-treatment Attachment to School. 

Interactions with achievement level. The 
only significant interaction of TESA partici- 
pation with achievement level was for the 
BSAP reading scale score in the within- 
School-A comparison for first grade. Table 
9 shows adjusted reading scale scores for 
low, medium, and high achievers in the first 
grade. It shows that the reading advantage 
for TESA participants over nonparticipants 
in School A (shown in Table 7) was solely 
due to the low achievers in the first grade. 
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The low-achieving first graders were the 
only group whose adjusted scores were 
higher than the scores of the comparison stu- 
dents. 

Observation data. As part of the TESA 
mtcrvention, teachers observed each others* 
classrooms and counted positive and nega- 
tive responses to students identified as low-, 
medium-, and high-achievers. A faithful 
implementation of TESA would show equal 
distributions of these teacher responses 
across the different achievement groupings. 
Table 10 compares these teacher responses 
for the three groups within each grade level. 

For first graders, the number of positive 
responses per student observation was 3.37 
for low and medium achievers and 5.12 for 
high achievers. For all achievement levels 
in this and all grade groupings, the number 
of negative responses per student observation 
was small and varied little across groups. 
The pattern of positive responses among 
grade 1 teachers looks like the pattern that 
would be expected in the absence of the 
TESA program: Lx)w achievers received 
fewer positive teacher responses. 

For grades two and three, the number of 
positive responses per student observation 
was 5.35 for low achievers, 4.71 for medi- 
um, and 4.55 for high achievers. This pat- 
tern of teacher responses - with more posi- 
tives provided for low achievers than for the 
other groups -- suggests that the teachers 
may have been overcompen sating in their 
responses. TESA seeks to produce an equal 
distribution of responses rather than to re- 
verse the direction of the inequality. 

Finally, for grade 5, medium achievers 
had a higher level of positive responses per 
student observation (4.74) than low achievers 



(4.33) or high achievers (3.79). The differ- 
ence between the highest and lowest group 
is roughly the same as the difference in 
grades 2 and 3, and the pattern of more 
positives for medium and low achievers 
again suggests teacher overcompensation. 

The absence of normative data and the 
small numbers of teachers for whom obser- 
vations are available limit the utility of these 
observation data. Nevertheless, they fail to 
show a consistent pattern of equal teacher 
responses across the different achievement 
levels. They also suggesi that the program 
was least well implemented at the first grade 
level. 



Conclusion 

This study provides little support for a 
positive effect of the TESA program, but 
this conclusion must be tempered with a 
methodological concern. 

When TESA participants within School 
A are compared with students in the class- 
rooms of non-TESA participants, no depend- 
able differences are found. A nonsignificant 
trend favors TESA participants in grades 1 
through 3. In grade 1, a nearly significant 
advantage on the r^^ading test score is due to 
better-than-expected performance of low 
achievers, as would be expected if TESA 
were well implemented. No such trend is 
found for grade 5 students. The observation 
data, however, fail to support the hypothesis 
that would be most consistent with this 
pattern of outcomes, i.e., that TESA was 
most faithfully implemented in grade 1 and 
least in grade 5. Instead, it appears that 
TESA was least well implemented by grade 
1 teachers. 



When TESA participants are compared 
with non-TESA participants from the com- 
parison school, the results favor the com- 
parison students. 

Our study does not enable us to differ- 
entiate among three inferences that might be 
drawn from the results. (1) The TESA 
program is slightly harmful, and the within- 
School A comparison is misleading because 
the teachers who volunteered for TESA 
participation were better teachers than those 
that did not volunteer. (2) The program has 
no effect, and the School A/School B com- 
parison is misleading because some more 
potent intervention occurred at School B 
which caused its students to perform better 
relative to those at School A. (3) The pro- 
gram was not well ininlemented and hence 
the study is incapable of evaluating its effec- 
tiveness. 

None of these inferences has strong 
support: (1) As noted earlier, ratings by the 
principal at School A and the TESA trainers 
did not differ significantly for the TESA and 
non-TESA teachers ~ a pattern inconsistent 
with the first of these interpretations. (2) 
None of the data we collected explains the 



unexpected observed differences between 
School A and School B. The Effective 
School Battery teacher survey results (Table 
4) and the student characteristics (Table 3) 
suggest that the schools were similar prior to 
the TESA program, and we know of no 
program, policy change, or staff change that 
would have created a beneficial change in 
School B during the year TESA was imple- 
mented in School A. (3) The pattern of 
teacher responses evident in the observation 
data suggests that the program may not have 
been well-implemented in School A, but 
these data are inconsistent with the percep- 
tions of the principal, teachers, and staff 
development personnel who thought the 
program standards were faithfully applied. 

The research design cannot controvert 
any of these competing explanations, so we 
are left with ambiguous results. In view of 
the popularity of this program, the limited 
extent of prior empirical support for its 
usefulness, and the ambiguity of our results, 
we conclude that further studies which 
include randomized trials and more careful 
records of the level of implementation are 
required. 
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Table 1 

Percentage Students Retained in 1989 by Race and Sex 



Group 


All 

fill 

CCSD 
schools' 

Wwl 1 WIW 


School A 
(TESA)'* 


School B 
fcomoarison^'* 


African-American males 


23.6 


21.4 


9.9 


African-American females 


15.7 


13.3 


10.4 


White males 


10.9 


5.1 


3.6 


Whitp fpmnlPQ 

f II MIC ICIIICIICO 


ft L 


0.0 


4.0 


Ratio of African-American to white 
retention rates: 








Males 


2.2 


4.2 


2.8 


Females 


1.9 


2.4 


2.3 



Source; Rose & Tee (1990). 

"Includes grades 1-8. 
"Includes grades 1-5. 
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Table 2 

Percentage of Students Not in the Grade Expected for Their Age, 
by Race and Sex -- Spring, 1989 



Group 


School A 
(TESA) 


School B 
(comparison) 


African-American males 


61.5 


46.8 


African-American females 


42.3 


42.7 


White males 


20.3 


19.5 


White females 


9.6 


6.5 




14 
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Table 3 

Selected Characteristics Measured in Spring, 1988 ■- TESA and Comparison Schools 



% meeting 

% % free- achievement 



School and grade 


African 

Enrollment American 


% not 
overage 


reduced 
lunch 


Standard 
Reading 


Math 


School A (TESA) 














K 


91 


41 


95 




— 


— 


1 


153 


39 


81 


27 


93 


89 


2 


156 


22 


75 


16 


96 


94 


3 


153 


35 


69 


24 


94 


92 


A 


137 


26 


75 








5 


122 


18 


69 








School B (comparison) 














K 


112 


25 


97 








1 


151 


39 


85 


26 


83 


86 


2 


140 


24 


81 


15 


94 


95 


3 


122 


25 


79 


14 


97 


95 


4 


114 


30 


70 








5 


127 


31 


77 


mm 




mm 



^Available only for grade levels included in state testing program. 
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Table 4 

Percentile Ranks for Teacher Scales from the Effective School Battery, 
Spring, 1988 



School A School B 

ESB scale (TESA) (comparison) 

Safety 94 99 

Morale 95 91 

Planning and Action 89 95 

Smooth Administration 94 87 

Resources 89 62 

Race Relations 92 91 

Parent/Community Involvement 99 99 

Student Influence 73 80 

Avoidance of the Use of Grades as a 86 93 
Sanction 

Pro-integration Attitudes 28 28 

Job Satisfaction 74 83 

Personal Security 89 76 

Classroom Orderliness 90 88 

Professional Development 84 71 

Nonauthoritarian Attitudes 41 45 

Number of teachers responding 46 28 

Percentage of teachers responding 88% 78% 
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Table 5 

Number and Percentage of Teachers and Students Participating in TESA - 
School A 



Grade level Students Teachers 



1 


63 


(36.0) 


3 


(33.3) 


2 


65 


(41.7) 


3 


(37.5) 


3 


77 


(48.4) 


4 


(50.0) 


4 


159 


(96.4) 


6 


(85.7) 


5 


101 


(69.6) 


4 


(66.7) 


Total 


465 


(58.1) 


20 


(52.6) 



Note. Some teachers teach more than one grade level. In these cases the 
teacher is included in the grade level in which most of the teacher's students are 
located. 
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Tables 

Means and Standard Deviations for All Pretreatment Measures, TESA and Comparison Groups, by Grade 



SclK>ol A School A 

TESA participants Non-participants Scliool B 

Pretreatment measure M SD N M SD N M SD N 



Grade 1 



Proportion male 


52 


.50 


63 


Proportion African American 


.35 


.48 


63 


Age 


6.34 


.50 


63 


Proportion free/reduced lunch 


.23 


.43 


60 


Proportion handicapped 


.06 


.24 


63 


Proportion in self-contained class 


.00 


.00 


63 


Proportion in resource 


.00 


.00 


63 


Proportion retained, Spring 1988 


.22 


.42 


63 


Total score, CSAB, Fall 1988 


101.05 


11.48 


63 


Grades 2 and 3 








Proportion male 


.48 


.50 


142 


Proportion African American 


.26 


.44 


142 


Age 


7.83 


.70 


142 


Proportion free/reduced lunch 


.17 


.38 


141 


Proportion handicapped 


.13 


.34 


142 


Proportion in self-contained class 


.00 


.00 


142 


Proportion in resource 


.01 


.08 


W. 


Proportion retained. Spring 1988 


.16 


.36 


4 ' 


Reading scale score, 


859.39 


72.53 


109 


BSAP, 1988 








Math scale score. 


821.38 


82.16 


109 



BSAP, 1988 
Grade 5 



Proportion male 


.52 


.50 


100 


Proportion African American 


.22 


.41 


100 


Age 


10.26 


.47 


100 


Proportion free/reduced lunch 


.10 


.30 


101 


Proportion handicapped 


.17 


.38 


100 


Proportion in self-contained class 


.00 


.00 


100 


Proportion in resource 


.10 


.30 


101 


Proportion retained. Spring 1988 


.26 


A\ 


101 


Total reading scale score. 


724.09 


44.64 


90 


CTBS, 1988 








Total math scale score. 


696.07 


29.24 


90 


CTBS. 1988 








Academic self-concept 


.00 


.43 


95 


Attachment to school 


.10 


.52 


96 


Teacher practices 


.11 


.53 


94 



.52 


.50 


112 


.58 


.50 


51 


.34 


.48 


112 


.31 


.47 


51 


6.29 


.54 


112 


6.39 


.56 


51 


.22 


.41 


102 


.18 


.39 


50 


.14 


.35 


112 


.20** 


.40 


51 


.01 


.09 


112 


.00 


.00 


51 


.07" 


.26 


112 


.00 


.00 


51 


.10" 


.30 


98 


.24 


.43 


51 


96.40" 


11.10 


101 


97.87 


10.00 


47 


.52 


.50 


172 


.50 


.50 


113 


.29 


.46 


172 


.32 


.47 


113 


7.87 


.80 


172 


7.86 


.77 


113 


.19 


.39 


162 


.17 


.38 


110 


.16 


.36 


172 


.10 


.30 


113 


.03" 


.18 


172 


.00 


.00 


113 


.06 


.24 


173 


AA*** 

.09 


AA 

.28 


114 


.05"* 


.22 


159 


.04*** 


.19 


109 


855.28 


81.93 


140 


A J A AA** 

846.92 


74.25 


AA 

93 


823.84 


82.41 


140 


836.41 


64.30 


93 


.68* 


.47 


44 


.54 


.50 


84 


.32 


.47 


44 


.26 


.44 


84 


10.45" 


.58 


44 


10.39* 


.68 


84 


.15 


.37 


26 


.14 


.34 


81 


.52"* 


.51 


44 


,06** 


.24 


84 


.34"* 


.48 


44 


.00 


.00 


83 


.09 


.29 


44 


.09 


.29 


85 


.04** 


.20 


26 


.05*** 


.22 


81 


717.96 


41.28 


23 


702.80*** 


49.36 


66 


706.91 


27.68 


23 


690.82 


21.51 


66 


.10 


.46 


23 


-.01 


.54 


69 


.11 


.62 


24 


.02* 


.52 


70 


.15 


.50 


24 


.02 


.55 


69 



*Mean of this group differs from TESA participant group mean, < .10. 
'*Mean of this group differs from TESA participant group mean, £ < .05. 
"Mean of this group differs from TESA participant group mean, £ < .01 . 
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Table? 

Means and Standard Deviations for All Outcome Measures, TESA and Comparison Schools, by Grade 



School A School A 

TESA participants Non-participants School B 



Outcome measure 


M 


SD 


N 


M 


SD 


N 


M 


SD 


N 


Grade 1 




















Proportion promoted, Spring 1989 


.84 


.37 


63 


.79 


.41 


103 


.90^ 


.30 


51 


Reading scale score, BSAP 


849.08 


86.38 


63 


840.14* 


98.91 


103 


856.35" 


86.94 


51 


Math scale score, BSAP 


831.81 


86.38 


63 


804.20 


94.64 


103 


828.72" 


83.40 


51 


Grades 2 and 3 




















Proportion promoted, Spring 1989 


.94 


.24 


142 


.86" 


.35 


173 


.91 


.28 


114 


Reading scale score, BSAP 


838.27 


83.13 


142 


847.31 


81.95 


163 


859.0r 


88.56 


113 


Math scale score, BSAP 


873.74 


100.40 


141 


857.39 


100.95 


163 


864.88 


109.74 


113 


Grades 


















Proportion promoted, Spring 1989 


.89 


.31 


100 


.92 


.28 


24 


.86 


.35 


85 


Total reading scale score, CTBS 


743.39 


39.08 


101 


733.15 


45.96 


26 


720.58"" 


44.16 


81 


Total math scale score, C TBS 


695.99 


101.00 


101 


707.42 


17.94 


26 


702.64 


16.77 


81 


Academic self-concept 


.04 


.47 


97 


.02 


.39 


25 


-.02 


.49 


69 


Attachment to school 


.14 


.55 


Q7 


.14 


.58 


25 


-.07"" 


.52 


69 


Teacher practices 


.18 


.56 


94 


-.01 


.53 


23 


.05 


.58 


69 



Note. Analysis of covariance adjusts for pre-existing differences on the following variables: grade 1 -proportion in resource, proportion 
retained (Spring 1988), total score CSAB (Fall 1988); grades 2 and 3"proportion in resource, proportion retained, and reading scale score 
BSAP (1988); grade S-age, proportion male, proportion handicapped, proportion retained (Spring 1988), total reading raw score CTBS (Fal 
1988). For Academic self-concept. Attachment to school, and Teacher practices, the respective pre-test measure vras included as a 
covariate. An asterisk indicates that the mean for that group differed from the mean for tho TESA participants. A letter indicates that the 
mean difference was significant once controls for pre-treatment differences were applied. 

'Mean of this group differs from TESA participant group mean, £ < .10. 
"Mean of this group differs from TESA participant group mean, £ < .05. 
"*Mean of this group differs from TESA participant group mean, £ < .01 . 

*Mean difference is significant, controlling on pretreatment measures, £ < .10. 
"Mean difference is significant, controlling on pretreatment measures, £ < .05. 
*Mean difference is significant, controlling on pretreatment measures, £ < .0 i . 
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Tables 

Adjusted Means for Outcome Variat)les for Which Significant Treatment Effects Were Found, TESA and 
Comparison Schools, by Grade 

School A School B 

Outcome measure TESA N Non-TESA N 



Grade 1 

Proportion promoted. Spring 1989 .84 63 .91 47 

Reading scale score, 849.08 63 864.08 47 

BSAP. 1989 

Math scale score, 821.81 63 833.63 47 

BSAP. 1989 

Grade 2 and 3 

Reading scale score, 842.78 108 868.94 3i 

BSAP. 1989 

Grades 

Total reading scale score, 744.94 89 719.11 65 

CTBS 

Attachment to school .18 87 -.08 52 



ERIC 



20 



Tables 

Mean BSAP Reading Scale Score, TESA and Comparison First Grades, by Achievement Level 



School A School A 

TESA participants Non-participants School B 



Achievement grouping 


Mean 


N 


Mean 


N 


Mean 


N 


High 


896 


32 


910 


27 


910 


15 


Medium 


822 


18 


866 


36 


877 


16 


Low 


771 


13 


753 


32 


808 


16 



Note. CSAB raw score and percent retained 'Spring, 1988) are controlled. Achievement level by TESA interaction is significant only for 
the School A TESA vs. non-TESA comparison. 
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Table 10 



Negative and Positive Responses to Students During Observations, by Student Achievement Level 



Achievement level 



Observation variable 



II students 


Low 


Medium 


High 


167 


34 


32 


101 


4 


2 


1 


4 


41.75 


17.00 


32.00 


25.25 


5.18 


3.37 


3.37 


5.12 


.01 


.01 


.01 


.00 


245 


68 


95 


82 


7 


5 


7 


7 


35.00 


13.60 


13.57 


11.71 


4.57 


5.35 


4.71 


4.55 


.01 


.00 


.01 


.00 


226 


CI 


34 


131 


5 


4 


4 


5 


45.20 


15.25 


8.50 


26.20 


4.05 


4.33 


4.74 


3.79 


.03 


.03 


.00 


.03 



Grade 1 
Number of observations 
Number of teachers 
Number of observations per teacher 
Positive responses per student observation 
Negative responses per student observation 

Grades 2 and 3 
Number of observations 
Number of teachers 
Number of observations per teacher 
Positive responses per student observation 
Negative responses per student observation 

Grade 5 
Number of observations 
Number cf teachers 
Number of observations per teacher 
Positive responses per student observation 
Negative responses per student observation 
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Figure 1 

Percentage of 1982-83 First Graders 
On- and Behind-Grade in 1988-89 



Percentage 



100 



80 



60 



40 



20 
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41.3 
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schools 



v.:.v.:.:.:.x.>x.:.;.;<.x.:.v.:.:.:.:. 
■:•^>^::^^^^^^^^.I:.:.xy:•^:•:!;•;•^x•;•:•:•x•^: 




^^^•.v.x•v.•.y.v.■.v.^y.X•:•X•X•X•!•:^•*■■ 

X':•:•:v^>:•;•:•.>v.v<,•.x.:.:.:•x•:.X':•^^:•:•:■ 
i^x^rWtWw':-:?^^ 

>•:•:¥:%W:%•:■:•^^:v^:•^^: 




School A 
(TESA) 



■•x«"«:-X':W'»x«x«>x«X'>x«X'X'*'*>'' 
■I'X-t't-X'X'XWW^'M'i'X'Xw:-:---- 
:::%W:«->::«:y>:%iA%%::¥^ 

;':->/.%x.:«::x«ox«'''*'X'X'' 
■ .•;■.■;■.■.■/•■ .•.•.■.•.•.•.■.■:v.v.x<«x*x«>*'"- 

•• v.v.v.X'X'J'X'X'X*' 
^^^^^^^^v^^X•.x■'••.;.■•v.y . 

■.■.•.•.y.::x<>X'X'X'X«''"'' 



33.3 



Behind grade 

On-grade 

No longer enrolled 



School B 
(comparison) 



ERIC 



9 



30 



Figure 2 

TESA INTERACTION MODEL 
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Source: Kerman, Kimball, & Martin (1980) . 



Table A-1 

Item Content of Attitude Scales 



Academic Self-concept 

The following items are coded "0"=Yes; "l"=No: 

1. I am satisHed with my school work. 

3. I am doing as well in school as I would like to. 
8. I am doing the best school work I can. 

10. I am proud of my school work. 

18. I am the kinl of person who will always be able to make it if I try. 
22. I like myself. 

The following items are coded "l"=Yes; "C"=No: 

2. I usually quit when my school work is too hard. 

4. I am a failure at school. 

5. Most boys and girls are smarter than I am. 

19. My teacher thinks that I am a slow learner. 

20. Sometimes I think I am no good at all. 

21. I feel I do not have much to be proud of. 

The following items are coded "0"=Very much; "r'=Somewhat; "2"=Not at all: 

How do most other students in your school see you? 

34. A good student? 

36. Successful? 

The following items are coded "2"=Very much; "l"=Somewhat; "0"=Not at all: 

How do most other students in your school see you? 

35. A trouble maker? 

37. A loser? 



Attachment to School 

The following items are coded "0"=Yes; "l"=No: 

6. I like to be called on by my teacher to answer questions. 

11. I wish I could have the same teacher next year. 

13. I am very happy when I am in school. 

14. My teacher listens to what I have to say. 
16. I like school very much. 

23. My teacher likes me. 



Note. All items were standardized and the items were averaged to create each scale. 
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The following items are coded "l"=Yes; "0"=No: 

7. I often feel like quitting school. 

9. My teacher feels that I am not good enough. 

12. Most of the time I do not want to go to school. 

15. My teacher likes some students more than others. 

17. My teacher does not like me to ask a lot of questions during the lesson. 

The following items are coded "0"=Always; "l"=Often; "2"=Sometimes; "3"=Seldom; 
•'4"=Never: 

24. I enjoy the work I do in class. 

25. I feel that I can go to my teacher with things that are on my mind. 

31. I can tell my teacher how I really feel about things. 

32. My teacher really knows how to get me to think. 



"ESA-Related Teacher Practices 

The following items are coded "0"=Always; "l"=Often; "2"=Sometimes; "3"=Seldom; 
"4"=Never: 

26. I am sure what Tm supposed to do in my class. 

27. My teacher gives me enough time to think before I give an answer. 

28. My teacher tells me when Tm doing the right thing and when Tm not. 

29. My teacher helps me think of the right answer by giving hints. 

30. I feel my teacher really listens to what I have to say. 
33. My teacher is polite to me. 

The following items are coded "0"=A lot; "l"=Sometimes; "2"=Not very often: 

38. My teacher calls on me. 

39. My teacher helps me. 

40. My teacher tells me when I am doing good schoolwork. 



i/e. All items were standardized and the items were averaged to create each scale. 



